The major function of a telecommunications (xe2x80x9ctelcoxe2x80x9d) network device such as a switch, router or hybrid switch/router is to transfer network data or packets between various physical interfaces or ports on the network device. The ports are connected to external network attachments, for example, optical fibers, coaxial cables and twisted pair cables. Typically, a network device includes multiple forwarding cards (i.e., printed circuit boards or modules) coupled with the external network attachments through the physical interfaces. Network packets are transferred between forwarding cards through a switch fabric.
Typically, network devices transfer network data with other network devices over the external network attachments in accordance with the same clock signal (i.e., synchronously). The clock signal may be a Building Integrated Supply (BITS) Line supplied to both network devices or one network device may use a timing signal provided by the other network device over a network attachment. Regardless of which clock signal is used, a central timing subsystem with the network device is generally used to distribute timing reference signals to all cards and components involved in synchronous data transfer, for example, all port cards. It is crucial that components and cards within the network device transfer data according to the same synchronized timing signals, as transferring data at different times, even slightly different times, may lead to data corruption, the wrong data being sent and/or a network device crash. Distributing clock signals, therefore, must be done carefully to insure that the clock signal received by each component is not skewed with respect to the clock signals received by other components.
Since timing distribution is critical to network device operation, network devices often include redundant central timing subsystems. Redundancy, further complicates the distribution of timing reference signals and increases the potential for skew. In addition, if a failure of the primary central timing subsystem is detected such that a switch over to the secondary central timing subsystem is initiated, it is important that the components receiving the timing reference signals do not experience sudden phase shifts in the timing reference signals from the two central timing subsystems, as phase shifts and other noise on clock signals may also lead to data corruption or a network device crash.
Many network devices also use an independent clock signal to transfer data through the internal switch fabric, and again, a switch fabric central timing system is often used to distribute the timing reference signals to components and/or cards involved with data transfer through the switch fabric. In addition, in network devices with multiple processors, processor timing reference signals are often sent from a processor central timing system to each processor to allow the processors to synchronize their processes. Distributing multiple timing reference signals consumes considerable routing resources in the network device and again increases the likelihood of skew between clock signals.
The present invention provides a network device including redundant, synchronous central timing subsystems (CTSs) and local timing subsystems (LTSs) including control circuits for automatically selecting between reference signals from both central timing subsystems. Automatically selecting between reference timing signals allows each LTS to quickly switchover from a failing or failed reference timing signal to a good reference timing signal. Quick switchovers prevent data corruption errors that may result during slow switchovers where a failing or failed reference signal is used for a longer period of time prior to the switch over. In addition, since each LTS independently monitors its received reference timing signals, a problem on one reference timing signal is quickly detected by the LTS and the LTS switches over to the good reference timing signal regardless of whether any other LTSs detect a problem with their received reference timing signals. Consequently, each LTS that detects a problem with a timing reference signal may immediately switch over to a good reference signal and not be forced to wait for all LTSs to be switched over.
In one aspect, the present invention provides a network device including a first central timing subsystem providing at least one first timing reference signal and at least one first master control signal, a second central timing subsystem providing at least one second timing reference signal and at least one second master control signal, where the first central timing subsystem receives the second timing reference signal and the second master control signal and the second central timing subsystem receives the first timing reference signal and the first master control signal and where the first central timing subsystem synchronizes the first timing reference signal to the second timing reference signal in accordance with the second master control signal and the second central timing subsystem synchronizes the second timing reference signal to the first timing reference signal in accordance with the first master control signal, and a local timing subsystem connected to the first timing reference signal and connected to the second timing reference signal and including a control circuit for selecting one of the first and second timing reference signals and for automatically selecting the other one of the first and second timing reference signals in accordance with a status detected on the selected one of the first and second timing reference signals. The control logic circuit automatically selects the other one of the first and second timing reference signals without software interaction. The status may be an invalid status or an inactive status. The status of the selected first timing reference signal may be a first status and the status of the selected second timing reference signal may be a second status and the local timing subsystem may further include a first status detector coupled with the control circuit for detecting the first status and a second status detector coupled with the control circuit for detecting the second status. The first status detector may be a first activity detector and the first status may be a first activity signal from the first activity detector, and the second status detector may be a second activity detector and the second status may be a second activity signal from the second status detector.
The local timing subsystem may further include a synchronization circuit for providing a local timing reference signal synchronous with the selected first or second timing reference signal. The synchronization circuit may include a phase locked loop circuit coupled with the selected first or second timing reference signal, and the control circuit may include a state machine. The local timing subsystem may be a first local timing subsystem and the network device may further include a second local timing subsystem connected to the first timing reference signal and connected to the second timing reference signal and including a control circuit for selecting one of the first and second timing reference signals and for automatically selecting the other one of the first and second timing reference signals in accordance with a status detected on the selected one of the first and second timing reference signal.
The first central timing subsystem may provide multiple first timing reference signals and the second central timing subsystem may provide multiple second timing reference signals and the local timing subsystem may be a first local timing subsystem connected with one of the first timing reference signals and one of the second timing reference signals and the network device may further include a second local timing subsystem connected to a different one of the first timing reference signals and connected to a different one of the second timing reference signals and include a control circuit for selecting one of the different first and second timing reference signals and for automatically selecting the other different first and second timing reference signal in accordance with a status detected on the selected different first and second timing reference signal. The control logic circuit may automatically select the other different one of the first and second timing reference signals without software interaction. The first central timing subsystem may provide multiple first timing reference signals and where at least two of the first timing reference signals comprise feedback inputs to the first central timing subsystem, where one of the feedback signals is routed on a printed circuit board on which the first central timing subsystem is located and the other feedback signal is routed off the printed circuit board and then back onto the printed circuit board.
In another aspect, the present invention provides a method of operating a network device including providing a first timing reference signal and a first master control signal from a first central timing subsystem, providing a second timing reference signal and a second master control signal from a second central timing subsystem, synchronizing the second timing reference signal to the first timing reference signal in accordance with the first master control signal, synchronizing the first timing reference signal to the second timing reference signal in accordance with the second master control signal, providing the first timing reference signal to a local timing subsystem, providing the second timing reference signal to the local timing subsystem, selecting one of the first and second timing reference signals within the local timing subsystem, detecting a status of the selected one of the first and second timing reference signals within the local timing subsystem, and selecting automatically the other one of the first and second timing reference signals within the local timing subsystem in accordance with the detected status. Automatically selecting the other one of the first and second timing reference signals is completed without software interaction. The method may further include synchronizing a local timing reference signal from the local timing subsystem to the selected one of the first and second timing reference signals, and synchronizing a local timing reference signal may include locking the local timing reference signal to the selected one of the first and second timing reference signals using a phase locked loop circuit. Detecting a status may include detecting an inactive status or an invalid status.
The local timing subsystem may be a first local timing subsystem and the method may further include providing the first timing reference signal to a second local timing subsystem, providing the second timing reference signal to the second local timing subsystem, selecting one of the first and second timing reference signals within the second local timing subsystem, detecting a status of the selected one of the first and second timing reference signals within the second local timing subsystem, and selecting automatically the other one of the first and second timing reference signals within the second local timing subsystem in accordance with the detected status. The local timing reference signal may be a first local timing reference signal and the method may further include synchronizing a second local timing reference signal from the second local timing subsystem to the selected one of the first and second timing reference signals within the second local timing subsystem.
In yet another aspect, the present invention provides a method of operating a network device including providing a first timing reference signal and a first master control signal from a first central timing subsystem, providing a second timing reference signal and a second master control signal from a second central timing subsystem, synchronizing the second timing reference signal to the first timing reference signal in accordance with the first master control signal, synchronizing the first timing reference signal to the second timing reference signal in accordance with the second master control signal, providing the first timing reference signal to a first local timing subsystem and to a second local timing subsystem, providing the second timing reference signal to the first and second local timing subsystems, selecting one of the first and second timing reference signals within the first local timing subsystem, and selecting a different one of the first and second timing reference signals within the second local timing subsystem. Providing a first timing reference signal may include providing multiple first timing reference signals and providing a second timing reference signal may include providing multiple second timing reference signals.
In a further aspect, the present invention provides a method of operating a network device including providing multiple first timing reference signals and a first master control signal from a first central timing subsystem, providing multiple second timing reference signals and a second master control signal from a second central timing subsystem, synchronizing the second timing reference signals to a first one of the first timing reference signals in accordance with the first master control signal, synchronizing the first timing reference signals to a first one of the second timing reference signals in accordance with the second master control signal, providing a second one of the first timing reference signals to a first local timing subsystem, providing a second one of the second timing reference signals to the first local timing subsystem, selecting one of the second one of the first and second timing reference signals within the first local timing subsystem, detecting a first status of the selected second one of the first and second timing reference signals within the first local timing subsystem, and selecting automatically the other second one of the first and second timing reference signals within the first local timing subsystem in accordance with the detected first status. The method may further include synchronizing a first local timing reference signal from the first local timing subsystem to the selected second one of the first and second timing reference signals, and the method may also include providing a third one of the first timing reference signals to a second local timing subsystem, providing a third one of the second timing reference signals to the second local timing subsystem, selecting one of the third one of the first and second timing reference signals within the second local timing subsystem, detecting a second status of the selected third one of the first and second timing reference signals within the second local timing subsystem, and selecting automatically the other third one of the first and second timing reference signals within the second local timing subsystem in accordance with the detected second status. The method may also include synchronizing a second local timing reference signal from the second local timing subsystem to the selected third one of the first and second timing reference signals.
In another aspect, the present invention provides a method of operating a network device including providing a plurality of first timing reference signals and a first master control signal from a first central timing subsystem, providing a plurality of second timing reference signals and a second master control signal from a second central timing subsystem, synchronizing the second timing reference signals to a first one of the first timing reference signals in accordance with the first master control signal, synchronizing the first timing reference signals to a first one of the second timing reference signals in accordance with the second master control signal, providing a second one of the first timing reference signals to a first local timing subsystem, providing a second one of the second timing reference signals to the first local timing subsystem, providing a third one of the first timing reference signals to a second local timing subsystem, providing a third one of the second timing reference signals to the third local timing subsystem, selecting the second one of the first timing reference signals within the first local timing subsystem, and selecting the third one of the second timing reference signals within the second local timing subsystem.